Graph Transformation Research Articles

Predicting associations between CircRNA and diseases through structure-aware graph transformer and path-integral convolution

A series of biological experiments has demonstrated that circular RNAs play a crucial regulatory role in cellular processes and may be potentially associated with diseases. Uncovering these connections helps in understanding potential disease mechanisms and advancing the development of treatment strategies. However, in biology, traditional experiments face limitations in terms of efficiency and cost, especially when enumerating possible associations. To address these limitations, several computational methods have been proposed, but existing methods only measure from a nodal perspective and cannot capture structural similarities between edges. In this study, we introduce an advanced computational method called SATPIC2CD for analyzing potential associations between circular RNAs and diseases. Specifically, we first employ an Structure-Aware Graph Transformer (SAT), which extracts five predefined metapath representations before calculating attention. This adaptive network integrates structural information into the original self-attention by aggregating information within and between paths. Subsequently, we use Path Integral Convolutional Networks (PACN) to integrate feature information for all path weights between two nodes. Afterward, we complement the network node features with feature loss and feature smoothing using Gated Recurrent Units (GRU) and node centrality. Finally, a Multi-Layer Perceptron (MLP) is employed to obtain the ultimate prediction scores for each circular RNA-disease pair. SATPIC2CD performs remarkably well, with an accuracy of up to 0.9715 measured by the Area Under the Curve (AUC) in a 5-fold cross-validation, surpassing other comparative models. Case studies further emphasize the high precision of our method in identifying circular RNA-disease associations, laying a solid foundation for guiding future biological research efforts.

The combination mode of forest and SVM for power network disaster response failure identification

The damage to power transmission and distribution equipment in different regions is inseparable from the erosion of natural and artificial disasters, affecting the power grid's regular power supply and users' typical power consumption. Therefore, research on failure risk prediction technology for power transmission and distribution equipment under different disasters has increasingly become critical to power grid reliability. Based on the combination mode of forest and Support Vector Machine (SVM), this study proposes a disaster response failure prediction model that considers the time cumulative effect of disaster elements. Based on this model, the failure degree indicators of different disaster elements are constructed by analyzing the relationship between disaster frequency and time. Then, the historical disaster data is used to calculate the index and form a training set with the monitoring data. The binary decision graph transformation method is combined with the adjacent node priority method to obtain the minimum cut set, and the SVM regression method is used to train to get the prediction model. The experimental results show that the effectiveness and accuracy of the proposed method are verified through the analysis of numerical examples and comparison of various techniques. The model's accuracy is 8.63 % higher than that of the traditional disaster prediction model, and the error rate of disaster failure prediction analysis is not more than 0.01.

Structural and positional ensembled encoding for Graph Transformer

In the Transformer architecture, positional encoding is a vital component because it provides the model with information about the structure and position of data. In Graph Transformer, there have been attempts to introduce different positional encodings and inject additional structural information. Therefore, in terms of integrating positional and structural information, we propose a Structural and Positional Ensembled Graph Transformer (SPEGT). We developed SPEGT by noting the different properties of structural and positional encodings of graphs and the similarity of their computational processes. We have set a unified component that integrates the functionalities: (i) Random Walk Positional Encoding, (ii) Shortest Path Distance between each node, and (iii) Hierarchical Cluster Encoding. We find a problem with a well-known positional encoding and experimentally verify that combining it with other encodings can solve their problem. In addition, SPEGT outperforms previous models on a variety of graph datasets. We also show that SPEGT using unified positional encoding, performs well on structurally indistinguishable graph data through error case analysis.

Multi-hop graph transformer network for 3D human pose estimation

Accurate 3D human pose estimation is a challenging task due to occlusion and depth ambiguity. In this paper, we introduce a multi-hop graph transformer network designed for 2D-to-3D human pose estimation in videos by leveraging the strengths of multi-head self-attention and multi-hop graph convolutional networks with disentangled neighborhoods to capture spatio-temporal dependencies and handle long-range interactions. The proposed network architecture consists of a graph attention block composed of stacked layers of multi-head self-attention and graph convolution with learnable adjacency matrix, and a multi-hop graph convolutional block comprised of multi-hop convolutional and dilated convolutional layers. The combination of multi-head self-attention and multi-hop graph convolutional layers enables the model to capture both local and global dependencies, while the integration of dilated convolutional layers enhances the model’s ability to handle spatial details required for accurate localization of the human body joints. Extensive experiments demonstrate the effectiveness and generalization ability of our model, achieving competitive performance on benchmark datasets.

Nonlinear slow-varying dynamics-assisted temporal graph transformer network for remaining useful life prediction

Remaining useful life (RUL) plays an important role in the prognostics and health management of mechanical systems. Recently, deep learning-based methods have been widely applied in the field of RUL prediction. However, there still suffer from two limitations. One is that the existing RUL prediction methods cannot capture spatial dependencies and long-term temporal dependencies. The other is that nonlinear slow-varying dynamics related to the degradation behavior have not been explored in the RUL prediction. To break these limitations, a nonlinear slow-varying dynamics-assisted temporal graph Transformer network (NSD-TGTN) is proposed in this paper for RUL prediction. NSD-TGTN can simultaneously capture and model spatiotemporal graphs and nonlinear slow-varying dynamics to achieve RUL prediction. Herein, the TGTN is developed to mine both spatial and long-term temporal dependencies for constructing the spatiotemporal features. And, nonlinear slow-varying features are built and introduced into the TGTN to enhance the RUL prediction capacity. Two datasets are utilized to validate the effectiveness and superiority of the proposed method. Compared with existing advanced methods, the average prediction accuracies of the NSD-TGTN on the C-MAPSS dataset and the wear dataset are improved by 1.70 % and 8.22 %, respectively.

GAT TransPruning: progressive channel pruning strategy combining graph attention network and transformer.

Recently, large-scale artificial intelligence models with billions of parameters have achieved good results in experiments, but their practical deployment on edge computing platforms is often subject to many constraints because of their resource requirements. These models require powerful computing platforms with a high memory capacity to store and process the numerous parameters and activations, which makes it challenging to deploy these large-scale models directly. Therefore, model compression techniques are crucial role in making these models more practical and accessible. In this article, a progressive channel pruning strategy combining graph attention network and transformer, namely GAT TransPruning, is proposed, which uses the graph attention networks (GAT) and the attention of transformer mechanism to determine the channel-to-channel relationship in large networks. This approach ensures that the network maintains its critical functional connections and optimizes the trade-off between model size and performance. In this study, VGG-16, VGG-19, ResNet-18, ResNet-34, and ResNet-50 are used as large-scale network models with the CIFAR-10 and CIFAR-100 datasets for verification and quantitative analysis of the proposed progressive channel pruning strategy. The experimental results reveal that the accuracy rate only drops by 6.58% when the channel pruning rate is 89% for VGG-19/CIFAR-100. In addition, the lightweight model inference speed is 9.10 times faster than that of the original large model. In comparison with the traditional channel pruning schemes, the proposed progressive channel pruning strategy based on the GAT and Transformer cannot only cut out the insignificant weight channels and effectively reduce the model size, but also ensure that the performance drop rate of its lightweight model is still the smallest even under high pruning ratio.

LogGT: Cross-system log anomaly detection via heterogeneous graph feature and transfer learning

Automated system log anomaly detection plays a crucial role in ensuring service reliability. Existing methods incompletely utilize structured log entries, resulting in the loss of key information such as components and time. Besides, due to the limitations of labeled data, models trained by a single system are difficult to apply to other systems. Therefore, we propose a cross-system log anomaly detection method named LogGT, which simultaneously models log events, components and time, leveraging labeled system knowledge to achieve anomaly detection in unlabeled systems. Specifically, we firstly design a heterogeneous graph to accurately represent the interactions between different events and components in the log sequence. Then, in order to avoid noise interference and conduct cross-system semantic analysis, we employ BERT to extract log sentence vectors, and Normalizing Flow is used to optimize them for smoother node embedding. A Graph Transformer Architecture with Time Intervals (GTAT) is proposed to model heterogeneous graphs by integrating time feature, allowing for a comprehensive analyze of execution order and time anomalies. Additionally, we design a semantic weighting method and utilize a novel domain-adapted transfer learning technology to effectively transfer the heterogeneous graph features of the source system to the target system. Experimental results demonstrate that LogGT outperforms five log anomaly detection methods, achieving an average anomaly detection F1-score higher than 0.95. Moreover, the AUC value of GTAT exceeds the sequence model by more than 2.3%.

Global contextual representation via graph-transformer fusion for hepatocellular carcinoma prognosis in whole-slide images

Current methods of digital pathological images typically employ small image patches to learn local representative features to overcome the issues of computationally heavy and memory limitations. However, the global contextual features are not fully considered in whole-slide images (WSIs). Here, we designed a hybrid model that utilizes Graph Neural Network (GNN) module and Transformer module for the representation of global contextual features, called TransGNN. GNN module built a WSI-Graph for the foreground area of a WSI for explicitly capturing structural features, and the Transformer module through the self-attention mechanism implicitly learned the global context information. The prognostic markers of hepatocellular carcinoma (HCC) prognostic biomarkers were used to illustrate the importance of global contextual information in cancer histopathological analysis. Our model was validated using 362 WSIs from 355 HCC patients diagnosed from The Cancer Genome Atlas (TCGA). It showed impressive performance with a Concordance Index (C-Index) of 0.7308 (95% Confidence Interval (CI): (0.6283–0.8333)) for overall survival prediction and achieved the best performance among all models. Additionally, our model achieved an area under curve of 0.7904, 0.8087, and 0.8004 for 1-year, 3-year, and 5-year survival predictions, respectively. We further verified the superior performance of our model in HCC risk stratification and its clinical value through Kaplan–Meier curve and univariate and multivariate COX regression analysis. Our research demonstrated that TransGNN effectively utilized the context information of WSIs and contributed to the clinical prognostic evaluation of HCC.

Deciphering cell types by integrating scATAC-seq data with genome sequences.

The single-cell assay for transposase-accessible chromatin using sequencing (scATAC-seq) technology provides insight into gene regulation and epigenetic heterogeneity at single-cell resolution, but cell annotation from scATAC-seq remains challenging due to high dimensionality and extreme sparsity within the data. Existing cell annotation methods mostly focus on the cell peak matrix without fully utilizing the underlying genomic sequence. Here we propose a method, SANGO, for accurate single-cell annotation by integrating genome sequences around the accessibility peaks within scATAC data. The genome sequences of peaks are encoded into low-dimensional embeddings, and then iteratively used to reconstruct the peak statistics of cells through a fully connected network. The learned weights are considered as regulatory modes to represent cells, and utilized to align the query cells and the annotated cells in the reference data through a graph transformer network for cell annotations. SANGO was demonstrated to consistently outperform competing methods on 55 paired scATAC-seq datasets across samples, platforms and tissues. SANGO was also shown to be able to detect unknown tumor cells through attention edge weights learned by the graph transformer. Moreover, from the annotated cells, we found cell-type-specific peaks that provide functional insights/biological signals through expression enrichment analysis, cis-regulatory chromatin interaction analysis and motif enrichment analysis.

Responsible composition and optimization of integration processes under correctness preserving guarantees

Enterprise Application Integration deals with the problem of connecting heterogeneous applications, and is the centerpiece of current on-premise, cloud and device integration scenarios. For integration scenarios, structurally correct composition of patterns into processes and improvements of integration processes are crucial. In order to achieve this, we formalize compositions of integration patterns based on their characteristics, and describe optimization strategies that help to reduce the model complexity, and improve the process execution efficiency using design time techniques. Using the formalism of timed DB-nets — a refinement of Petri nets — we model integration logic features such as control- and data flow, transactional data storage, compensation and exception handling, and time aspects that are present in reoccurring solutions as separate integration patterns. We then propose a realization of optimization strategies using graph rewriting, and prove that the optimizations we consider preserve both structural and functional correctness. We evaluate the improvements on a real-world catalog of pattern compositions, containing over 900 integration processes, and illustrate the correctness properties in case studies based on two of these processes.

Graph Node Classification to Predict Autism Risk in Genes.

This study explores the genetic risk associations with autism spectrum disorder (ASD) using graph neural networks (GNNs), leveraging the Sfari dataset and protein interaction network (PIN) data. We built a gene network with genes as nodes, chromosome band location as node features, and gene interactions as edges. Graph models were employed to classify the autism risk associated with newly introduced genes (test set). Three classification tasks were undertaken to test the ability of our models: binary risk association, multi-class risk association, and syndromic gene association. We tested graph convolutional networks, Graph Sage, graph transformer, and Multi-Layer Perceptron (Baseline) architectures on this problem. The Graph Sage model consistently outperformed the other models, showcasing its utility in classifying ASD-related genes. Our ablation studies show that the chromosome band location and protein interactions contain useful information for this problem. The models achieved 85.80% accuracy on the binary risk classification, 81.68% accuracy on the multi-class risk classification, and 90.22% on the syndromic classification.

SGT++: Improved Scene Graph-Guided Transformer for Surgical Report Generation.

Automatically recording surgical procedures and generating surgical reports are crucial for alleviating surgeons' workload and enabling them to concentrate more on the operations. Despite some achievements, there still exist several issues for the previous works: 1) failure to model the interactive relationship between surgical instruments and tissue; and 2) neglect of fine-grained differences within different surgical images in the same surgery. To address these two issues, we propose an improved scene graph-guided Transformer, also named by SGT++, to generate more accurate surgical report, in which the complex interactions between surgical instruments and tissue are learnt from both explicit and implicit perspectives. Specifically, to facilitate the understanding of the surgical scene graph under a graph learning framework, a simple yet effective approach is proposed for homogenizing the input heterogeneous scene graph. For the homogeneous scene graph that contains explicit structured and fine-grained semantic relationships, we design an attention-induced graph transformer for node aggregation via an explicit relation-aware encoder. In addition, to characterize the implicit relationships about the instrument, tissue, and the interaction between them, the implicit relational attention is proposed to take full advantage of the prior knowledge from the interactional prototype memory. With the learnt explicit and implicit relation-aware representations, they are then coalesced to obtain the fused relation-aware representations contributing to generating reports. Some comprehensive experiments on two surgical datasets show that the proposed STG++ model achieves state-of-the-art results.

Dynamic graph transformer network via dual-view connectivity for autism spectrum disorder identification

Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder that requires objective and accurate identification methods for effective early intervention. Previous population-based methods via functional connectivity (FC) analysis ignore the differences between positive and negative FCs, which provide the potential information complementarity. And they also require additional information to construct a pre-defined graph. Meanwhile, two challenging demand attentions are the imbalance of performance caused by the class distribution and the inherent heterogeneity of multi-site data. In this paper, we propose a novel dynamic graph Transformer network based on dual-view connectivity for ASD Identification. It is based on the Autoencoders, which regard the input feature as individual feature and without any inductive bias. First, a dual-view feature extractor is designed to extract individual and complementary information from positive and negative connectivity. Then Graph Transformer network is innovated with a hot plugging K-Nearest Neighbor (KNN) algorithm module which constructs a dynamic population graph without any additional information. Additionally, we introduce the PolyLoss function and the Vrex method to address the class imbalance and improve the model's generalizability. The evaluation experiment on 1102 subjects from the ABIDE I dataset demonstrates our method can achieve superior performance over several state-of-the-art methods and satisfying generalizability for ASD identification.

Functional Connectivity Prediction With Deep Learning for Graph Transformation.

Inferring resting-state functional connectivity (FC) from anatomical brain wiring, known as structural connectivity (SC), is of enormous significance in neuroscience for understanding biological neuronal networks and treating mental diseases. Both SC and FC are networks where the nodes are brain regions, and in SC, the edges are the physical fiber nerves among the nodes, while in FC, the edges are the nodes' coactivation relations. Despite the importance of SC and FC, until very recently, the rapidly growing research body on this topic has generally focused on either linear models or computational models that rely heavily on heuristics and simple assumptions regarding the mapping between FC and SC. However, the relationship between FC and SC is actually highly nonlinear and complex and contains considerable randomness; additional factors, such as the subject's age and health, can also significantly impact the SC-FC relationship and hence cannot be ignored. To address these challenges, here, we develop a novel SC-to-FC generative adversarial network (SF-GAN) framework for mapping SC to FC, along with additional metafeatures based on a newly proposed graph neural network-based generative model that is capable of learning the stochasticity. Specifically, a new graph-based conditional generative adversarial nets model is proposed, where edge convolution layers are leveraged to encode the graph patterns in the SC in the form of a graph representation. New edge deconvolution layers are then utilized to decode the representation back to FC. Additional metafeatures of subjects' profile information are integrated into the graph representation with newly designed sparse-regularized layers that can automatically select features that impact FC. Finally, we have also proposed new post hoc explainer of our SF-GAN, which can identify which subgraphs in SC strongly influence which subgraphs in FC by a new multilevel edge-correlation-guided graph clustering problem. The results of experiments conducted to test the new model confirm that it significantly outperforms existing state-of-the-art methods, with additional interpretability for identifying important metafeatures and subgraphs.

Enhanced HAR using Dynamic STGAT

Action recognition has seen significant advancements with the integration of spatio-temporal representations, particularly leveraging skeleton-based models and cross-modal data fusion techniques. However, existing approaches face challenges in capturing long- range dependencies within the human body skeleton and effectively balancing features from diverse modalities. To address these limitations, a novel framework, the Dynamic Spatio-Temporal Graph Attention Transformer (D-STGAT), is proposed, which seamlessly integrates the strengths of dynamic graph attention mechanisms and transformer architectures for enhanced action recognition. The framework builds upon recent innovations in graph attention networks (GAT) and transformer models. First, the Spatial-Temporal Dynamic Graph Attention Network (ST-DGAT) is introduced, extending traditional GAT by incorporating a dynamic attention mechanism to capture spatial- temporal patterns within skeleton sequences. By reordering the weighted vector operations in GAT, the approach achieves a global approximate attention function, significantly enhancing its expressivity and capturing long-distance dependencies more effectively than static attention mechanisms. Furthermore, to address the challenges of cross-modal feature representation and fusion, the spatio-temporal Cross Attention Transformer (ST-CAT) is introduced. This model efficiently integrates spatio-temporal information from both video frames and skeleton sequences by employing a combination of full spatio-temporal attention (FAttn), zigzag spatio-temporal attention (ZAttn), and binary spatio-temporal attention (BAttn) modules. Through the proper arrangement of these modules within the transformer encoder and decoder, ST-CAT learns a multi-feature representation that effectively captures the intricate spatiotemporal dynamics inherent in action recognition tasks. Experimental results on the Penn- Action, NTU-RGB+D 60, and 120 datasets showcase the efficacy of the approach, yielding promising performance improvements over previous state-of-the-art methods. In summary, the proposed D-STGAT and ST-CAT frameworks offer novel solutions for action recognition tasks by leveraging dynamic graph attention mechanisms and transformer architectures to effectively capture and fuse spatiotemporal features from diverse modalities, leading to superior performance compared to existing approaches.

JUST-IN-TIME VULNERABILITY DETECTION AND LOCALIZATION

Software vulnerabilities have increased dramatically, and multiple severe attacks have occurred in recent years. This poses a critical challenge for early detection and prevention of vulnerabilities in Software Quality Assurance. This paper introduces a novel framework, JULY, which serves the dual purpose of detecting vulnerable commits and localizing the root causes of the vulnerabilities. The fundamental concept of JULY is that the determinant of the vulnerability of a commit is the inherent meaning embedded in its changed code. For just-in-time vulnerability detection (JIT-VD ), JULY represents each commit by a Code Transformation Graph and employs a Graph Neural Network model to capture their meanings and distinguish between vulnerable and non-vulnerable commits. Once a commit is detected as vulnerable, it is passed to the just-in-time vulnerability localization (JIT-VL ) model to localize the root causes, which are vulnerable changed statements. In JIT-VL , JULY encodes each statement by the following features: operation, context, and topic. Then, JULY measures the suspiciousness score of each changed statement and ranks them based on their scores. To evaluate the effectiveness of JULY, we conducted several experiments using a dataset consisting of 20,274 commits in 506 C/C++ projects. JULY achieves a remarkable improvement of 95% in Top-1 ACC and 63% in MRR compared to the state-of-the-art approaches. Furthermore, when examining the same portion (i.e., 20%) of modified statements in each commit, JULY can find twice as many vulnerable statements within a given commit as the state-of-the-art approaches.      

Exploring sparsity in graph transformers

Graph Transformers (GTs) have achieved impressive results on various graph-related tasks. However, the huge computational cost of GTs hinders their deployment and application, especially in resource-constrained environments. Therefore, in this paper, we explore the feasibility of sparsifying GTs, a significant yet under-explored topic. We first discuss the redundancy of GTs based on the characteristics of existing GT models, and then propose a comprehensive Graph Transformer SParsification (GTSP) framework that helps to reduce the computational complexity of GTs from four dimensions: the input graph data, attention heads, model layers, and model weights. Specifically, GTSP designs differentiable masks for each individual compressible component, enabling effective end-to-end pruning. We examine our GTSP through extensive experiments on prominent GTs, including GraphTrans, Graphormer, and GraphGPS. The experimental results demonstrate that GTSP effectively reduces computational costs, with only marginal decreases in accuracy or, in some instances, even improvements. For example, GTSP results in a 30% reduction in Floating Point Operations while contributing to a 1.8% increase in Area Under the Curve accuracy on the OGBG-HIV dataset. Furthermore, we provide several insights on the characteristics of attention heads and the behavior of attention mechanisms, all of which have immense potential to inspire future research endeavors in this domain. Our code is available at https://github.com/LiuChuang0059/GTSP.

Transitivity-Preserving Graph Representation Learning for Bridging Local Connectivity and Role-Based Similarity

Graph representation learning (GRL) methods, such as graph neural networks and graph transformer models, have been successfully used to analyze graph-structured data, mainly focusing on node classification and link prediction tasks. However, the existing studies mostly only consider local connectivity while ignoring long-range connectivity and the roles of nodes. In this paper, we propose Unified Graph Transformer Networks (UGT) that effectively integrate local and global structural information into fixed-length vector representations. First, UGT learns local structure by identifying the local sub-structures and aggregating features of the k-hop neighborhoods of each node. Second, we construct virtual edges, bridging distant nodes with structural similarity to capture the long-range dependencies. Third, UGT learns unified representations through self-attention, encoding structural distance and p-step transition probability between node pairs. Furthermore, we propose a self-supervised learning task that effectively learns transition probability to fuse local and global structural features, which could then be transferred to other downstream tasks. Experimental results on real-world benchmark datasets over various downstream tasks showed that UGT significantly outperformed baselines that consist of state-of-the-art models. In addition, UGT reaches the third-order Weisfeiler-Lehman power to distinguish non-isomorphic graph pairs.

Span Graph Transformer for Document-Level Named Entity Recognition

Named Entity Recognition (NER), which aims to identify the span and category of entities within text, is a fundamental task in natural language processing. Recent NER approaches have featured pre-trained transformer-based models (e.g., BERT) as a crucial encoding component to achieve state-of-the-art performance. However, due to the length limit for input text, these models typically consider text at the sentence-level and cannot capture the long-range contextual dependency within a document. To address this issue, we propose a novel Span Graph Transformer (SGT) method for document-level NER, which constructs long-range contextual dependencies at both the token and span levels. Specifically, we first retrieve relevant contextual sentences in the document for each target sentence, and jointly encode them by BERT to capture token-level dependencies. Then, our proposed model extracts candidate spans from each sentence and integrates these spans into a document-level span graph, where nested spans within sentences and identical spans across sentences are connected. By leveraging the power of Graph Transformer and well-designed position encoding, our span graph can fully exploit span-level dependencies within the document. Extensive experiments on both resource-rich nested and flat NER datasets, as well as low-resource distantly supervised NER datasets, demonstrate that proposed SGT model achieves better performance than previous state-of-the-art models.

TextGT: A Double-View Graph Transformer on Text for Aspect-Based Sentiment Analysis

Aspect-based sentiment analysis (ABSA) is aimed at predicting the sentiment polarities of the aspects included in a sentence instead of the whole sentence itself, and is a fine-grained learning task compared to the conventional text classification. In recent years, on account of the ability to model the connectivity relationships between the words in one sentence, graph neural networks have been more and more popular to handle the natural language processing tasks, and meanwhile many works emerge for the ABSA task. However, most of the works utilizing graph convolution easily incur the over-smoothing problem, while graph Transformer for ABSA has not been explored yet. In addition, although some previous works are dedicated to using both GNN and Transformer to handle text, the methods of tightly combining graph view and sequence view of text is open to research. To address the above issues, we propose a double-view graph Transformer on text (TextGT) for ABSA. In TextGT, the procedure in graph view of text is handled by GNN layers, while Transformer layers deal with the sequence view, and these two processes are tightly coupled, alleviating the over-smoothing problem. Moreover, we propose an algorithm for implementing a kind of densely message passing graph convolution called TextGINConv, to employ edge features in graphs. Extensive experiments demonstrate the effectiveness of our TextGT over the state-of-the-art approaches, and validate the TextGINConv module. The source code is available at https://github.com/shuoyinn/TextGT.